A Probabilistic Analysis For Fault Detectability of Code Coverage Metrics

Increasing design complexity, shrinking time to market, and high cost of fixing a bug in a released product [11] make functional validation of microprocessors a key ingredient in the product development cycle. Simulation is widely used to validate large systems like microprocessors. In simulation based validation, a test is executed in a golden reference model as well as in the design under test (DUT), which in this case is the RTL implementation of the microprocessor. A bug is recorded if a difference in the state of the golden model and the DUT is detected. Test generation for simulation based validation can be broadly classified into exhaustive, random, and coverage directed test generation. Exhaustive testing is guaranteed to catch all bugs in the implementation but is not practical due to its exponential complexity. Random test generators [2], [5] are extensively used to validate microprocessors, but they are found to generate test cases that exercise the same feature of the processor over and over again. Coverage directed test generation techniques [6], [7], [10], [12] create test suites that satisfy the coverage metric and they differ in the coverage metric being targeted and the technique used to generate test cases. The effectiveness of coverage directed test generation in finding a bug depends on the chosen coverage metric and the type of bug present in the design. In our previous work [3], we had analyzed the descriptions of the modeling errors reported in microprocessor validation study conducted by Velev [13] and Campenhout [4] and related them to the fault models proposed in software validation by Lau and Yu [9]. We found a high correlation between the fault models in software validation and the modeling errors, as shown in Table 1. For example, 56.8% of the bugs found by Velev can be categorized into Term Omission Fault (TOF). Therefore, we concluded that the fault classes presented by Lau and Yu [9] for software validation is a good categorization for the bugs typically seen during microprocessor validation.